This invention relates to an apparatus and method for coding moving pictures. More particularly, the invention relates to a moving-picture coding apparatus and method for suppressing a decline in image quality that accompanies loss of coded data caused by occurrence of an error during transmission.
In the conventional apparatus and method for coding moving pictures, intraframe coding or interframe predictive coding is selected and used adaptively in units of small blocks.
FIG. 8 is a flowchart useful in describing the processing procedure of a prior-art control method for selecting the small blocks in which intraframe coding is forcibly performed. When coding is initiated following the start of processing of a moving picture sequence, first a referential forced refresh map is reset in order to select a small block in which intraframe coding is to be forcibly performed (S21). A value indicating the priority at which forced refresh is to be performed is recorded in the map for every small block contained in the image frame.
In the coding of each frame of a moving picture, coding is carried out upon selecting either intraframe coding or interframe predictive coding for each small block (S22 to S30). Before the selection is made, it is determined at S23 whether the small block of interest is a candidate for forced refresh. If the decision rendered is xe2x80x9cYESxe2x80x9d at S23, intraframe coding is selected (S24); if xe2x80x9cNOxe2x80x9d, then either intraframe coding or interframe predictive coding, whichever provides the higher coding efficiency, is selected (S26). If the intraframe coding mode has been selected for the small block desired to be coded (xe2x80x9cYESxe2x80x9d at S23), then the forced refresh priority of this block is reset to the lowest level (S25), followed by updating of the forced refresh map (S29).
If interframe predictive coding is selected at S26, the power of prediction error for a case where image data at the same position in a reference frame is adopted as the prediction data is calculated. If this value is greater than a predetermined threshold value, then the forced refresh priority of the block is updated to a higher level (S28), followed by updating of the forced refresh map (S29).
When the processing for coding the small block is completed through the foregoing procedure (S30), control returns to S22 for the processing of the next small block.
FIG. 9 is a block diagram illustrating a moving-picture coding system for implementing the method according to the prior art described above. According to this example of the prior art, the coding system employs motion-compensating interframe prediction. When motion is detected from one frame to the next, the power of frame-to-frame prediction error which prevails when the motion is made zero is actually measured. Accordingly, the value of forced refresh priority is re-calculated for each small block upon referring to interframe prediction error data. Further, if the intraframe coding mode has been selected for a small block of interest, the value of the forced refresh map is reset by a control signal 902.
FIG. 10 is a diagram useful in describing an example of an error suppression method on the decoding side. This diagram conceptually illustrates a method assumed to employ the coding scheme of the prior art described above. In FIG. 10, image data is interpolated by interframe predictive processing assuming that motion and prediction error are zero in small blocks in which data has been lost due to error during transmission. A block expected to have conspicuously poor image quality because this interpolation processing is inappropriate has the priority of its forced refresh raised by the method described above. In accordance with the conventional method, therefore, blocks whose image quality will decline conspicuously owing to loss of data are selected and forced refresh is applied giving priority to these blocks.
However, certain problems have been encountered in the course of investigations toward the present invention. Namely with the conventional method described above, a problem has been encountered that coding efficiency per se declines when forced refresh is inserted frequently to sufficiently suppress a decline in image quality caused by transmission error. The reason for this is that with the conventional method described above, the calculation of forced refresh priority for each small block does not take into account the fact that the probability of data loss due to transmission error differs from one small block to the next. As a consequence, even small blocks having a low probability of data loss are updated just as frequently as small blocks for which the probability of data loss is high, and the intraframe coding mode, for which the coding efficiency is not always the best, is selected often. In other words, if the frequency with which forced refresh is applied is raised in accordance with the small blocks having the higher probability of data loss, small blocks for which the probability of data loss is low will be updated unnecessarily often. Conversely, if the frequency of updating is made to conform to lower probability, there is a danger that small blocks for which the frequency of data loss is high will remain without being refreshed for an extended period of time during which image quality declines due to error.
Further, with the method according to the prior art, small blocks that are to undergo forced refresh are selected upon referring to the power of simple interframe prediction error. This is because only suppression of error resulting from simple interframe prediction is assumed on the decoding side as well. Accordingly, it is difficult to adjust the frequency of forced refresh and raise coding efficiency also in cases where a more sophisticated error suppression method is employed on the decoding side.
Accordingly, an object of the present invention is to provide a moving-picture coding apparatus and method through which a conspicuous decline in image quality is avoided without lowering coding efficiency even in an environment in which error occurs in coded data during transmission.
According to an aspect of the present invention, there is provided a moving-picture coding apparatus for adaptively selecting and using intraframe coding or interframe predictive coding in small blocks. The apparatus comprises: data-loss probability estimating means for counting, per small block, distance from a synchronizing pattern that is inserted into coded data, and estimating, from the count, data-loss probability that coded data of a target small block will be lost during transmission thereof;
image-quality degradation estimating means for estimating, in regard to a small block in which interframe predictive coding has been performed, degree of image-quality degradation that will be caused in this small block in a decoded image in a case where data has been lost during transmission thereof;
a forced refresh map for calculating a forced refresh priority of each small block upon comparing the estimated data-loss probability and estimated degree of image-quality degradation with threshold values, and recording a value of priority calculated for each small block of an entire image frame; and
mode control means for referring to recorded values of priority when a subsequent frame is to be coded and selecting a small block in which intraframe coding is to be forcibly performed based upon the recorded values of priority.
The moving-picture coding apparatus may further comprise a counter that counts the distance from the synchronizing pattern per small block, wherein the data-loss probability estimating means calculates an estimated value of data-loss probability as a monotonously increasing function of the value of distance based upon the count output by the counter.
The counter may count, as the value of distance, amount of code or number of coded blocks between the synchronizing pattern and a target small block, wherein the data-loss probability estimating means calculates estimated probability of data loss as a monotonously increasing function of the amount of code or number of coded blocks.
In estimating the degree of image-quality degradation that will be caused in the above-mentioned small block in a decoded image in a case where the coded data of this small block in which interframe predictive coding has been performed has been lost during transmission, the degree of image-quality degradation may be calculated from any one of: (a) power of prediction error in a case where the target small block has been subjected to simple interframe prediction, (b) power of prediction error in a case where the target small block has been subjected to motion-compensated interframe prediction, and magnitude of a motion vector used in this motion compensation, and (c) power of prediction error in a case where the target small block has been subjected to motion-compensated interframe prediction, and magnitude of a difference vector between a motion vector used in this motion compensation and a motion vector of an adjacent small block.
Further, according to a second aspect of the present invention, there is provided a moving-picture coding method for adaptively selecting and using intraframe coding or interframe predictive coding in small blocks. The method comprises:
a data-loss probability estimating step of counting, per small block, distance from a synchronizing pattern that is inserted into coded data, and estimating, from the count, data-loss probability that coded data of a target small block will be lost during transmission;
an image-quality degradation estimating step of estimating, in regard to a small block in which interframe predictive coding has been performed, degree of image-quality degradation that will be caused in this small block in a decoded image in a case where the coded data has been lost during transmission;
a forced refresh map creation step of calculating a forced refresh priority of each small block upon comparing the estimated data-loss probability and estimated degree of image-quality degradation with threshold values, and recording a value of priority calculated for each small block of an entire image frame; and
a mode control step of referring to recorded values of priority when a subsequent frame is to be coded and selecting a small block in which intraframe coding is to be forcibly performed based upon the recorded values of priority.
The data-loss probability estimating step may calculate an estimated value of data-loss probability as a monotonously increasing function of the value of distance based upon a value obtained by counting, per small block, the distance from the synchronizing pattern.
The count serving as the value of distance may be obtained by counting amount of code or number of coded blocks between the synchronizing pattern and a target small block, wherein the data-loss probability estimating step calculates estimated probability of data loss as a monotonously increasing function of the amount of code or number of coded blocks.
In estimating the degree of image-quality degradation that will be caused in the above-mentioned small block in a decoded image in a case where the coded-data of this small block in which interframe predictive coding has been performed has been lost during transmission, the degree of image-quality degradation may be calculated from any one of: (a) power of prediction error in a case where the target small block has been subjected to simple interframe prediction, (b) Power of prediction error in a case where the target small block has been subjected to motion-compensated interframe prediction, and magnitude of a motion vector used in this motion compensation, and (c) power of prediction error in a case where the target small block has been subjected to motion-compensated interframe prediction, and magnitude of a difference vector between a motion vector used in this motion compensation and a motion vector of an adjacent small block.
Other features and advantages of the present invention will be apparent from the following description taken in conjunction with the accompanying drawings, in which like reference characters designate the same or similar parts throughout the figures thereof.